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Preface 


This  short  paper  seeks  to  replicate  and  update  a  similar  RAND  effort  published  in  2011, 
Counterinsurgency  Scorecard:  Afghanistan  in  Early  2011  Relative  to  the  Insurgencies  of  the  Past 
30  Years  (OP-337-OSD).  The  core  of  the  research  involved  conducting  an  expert  elicitation 
exercise  (using  classic  Delphi  methods)  to  complete  the  counterinsurgency  (COIN)  scorecard 
developed  as  part  of  another  study,  the  findings  from  which  were  published  in  Victory  Has  a 
Thousand  Fathers:  Sources  of  Success  in  Counterinsurgency  (MG-964-OSD).  The  scorecard  was 
updated,  based  on  an  expanded  case  set,  for  the  2013  report  Paths  to  Victory:  Lessons  from 
Modern  Insurgencies  (RR-291/1-OSD).  The  expert  elicitation  recounted  here  was  completed  in 
April  and  May  2013. 

The  goal  of  this  effort  was  to  apply  methods  developed  to  examine  historical  insurgencies 
to  assess  the  progress  and  the  prospects  for  success  of  ongoing  COIN  efforts  in  Afghanistan  as 
of  early  2013. 

This  work  will  be  of  interest  to  defense  analysts  and  military  planners  who  are  responsible 
for  evaluating  current  U.S.  operations  and  COIN  approaches,  particularly  in  Afghanistan;  to 
military  and  civilian  decisionmakers  with  responsibility  for  Afghanistan;  to  academics  and 
scholars  who  engage  in  historical  research  of  COIN,  insurgency,  and  irregular  warfare;  and  to 
students  of  contemporary  and  historic  international  conflicts. 

This  research  was  sponsored  by  the  Office  of  the  Secretary  of  Defense,  Cost  Assessment 
and  Program  Evaluation,  and  conducted  within  the  International  Security  and  Defense  Policy 
Center  of  the  RAND  National  Defense  Research  Institute,  a  federally  funded  research  and 
development  center  sponsored  by  the  Office  of  the  Secretary  of  Defense,  the  Joint  Staff,  the 
Unified  Combatant  Commands,  the  Navy,  the  Marine  Corps,  the  defense  agencies,  and 
the  defense  Intelligence  Community. 

For  more  information  on  the  International  Security  and  Defense  Policy  Center,  see  http:// 
www.rand.org/nsrd/ndri/centers/isdp.html  or  contact  the  director  (contact  information  is  pro¬ 
vided  on  the  web  page). 
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Summary 


The  RAND  report  Paths  to  Victory :  Lessons  from  Modern  Insurgencies  used  detailed  case  studies 
of  71  insurgencies  worldwide  since  World  War  II  to  analyze  correlates  of  success  in  counter¬ 
insurgency  (COIN).  One  of  the  core  findings  of  that  effort  was  that  a  case’s  score  on  a  scorecard 
of  15  good  factors  or  practices  minus  11  bad  factors  or  practices  perfectly  discriminated  the 
outcomes  of  the  59  core  cases  in  the  data.1  Cases  in  which  COIN  forces  were  able  to  maximize 
the  presence  of  good  factors  and  minimize  the  presence  of  bad  factors  resulted  in  COIN  force 
success.  Specifically,  cases  with  a  good-minus-bad  score  of  +2  or  greater  were  always  won  by 
the  government,  and  cases  with  a  good-minus-bad  score  of  -1  or  lower  were  always  won 
by  the  insurgents.2  The  research  found  that  all  successful  COIN  forces  have  a  scorecard  score 
of  at  least  +2  and  that  all  successful  COIN  campaigns  realized  three  specific  factors:  the  dis¬ 
ruption  of  tangible  support  to  the  insurgents,  the  demonstration  of  commitment  and  motiva¬ 
tion  on  the  part  of  both  the  government  and  COIN  forces,  and  flexibility  and  adaptability  on 
the  part  of  COIN  forces. 

The  current  research  effort  involved  developing  and  conducting  an  expert  elicitation  exer¬ 
cise  to  complete  the  scorecard  for  operations  in  Afghanistan  in  early  2013.  We  used  an  expert 
elicitation  for  two  reasons.  First,  much  of  the  available  data  on  contemporary  Afghanistan 
leave  something  to  be  desired  in  terms  of  reliability  and  validity.  Discussion  moving  toward 
consensus  among  a  panel  of  experts — many  of  whom  have  spent  time  in  Afghanistan — is 
methodologically  stronger  than  claiming  certainty  based  on  our  own  interpretation  of  tenuous 
data.  Second,  assessments  of  Afghanistan  are  potentially  politically  contentious,  and  an  anony¬ 
mous  panel  helps  shield  the  analysis  from  some  of  that  contention. 

We  asked  a  panel  of  19  experts  on  Afghanistan  to  make  “worst-case”  assessments  of  the 
scorecard  factors.  According  to  the  consensus  results  for  the  scorecard,  eight  good  factors  and 
six  bad  factors  were  present  in  early  2013  Afghanistan,  for  an  overall  score  of  +2.  This  repre¬ 
sents  an  improvement  over  the  score  identified  through  a  similar  process  in  early  2011.  A  total 
score  of  +2  is  equal  to  that  of  the  lowest-scoring  winning  historical  COIN  forces,  and  having 
eight  good  factors  is  also  consonant  with  historical  success.  However,  only  cases  in  which  the 
insurgents  prevailed  had  as  many  as  six  bad  factors.  Also  concerning  is  the  repeated  observation 


1  The  study  considered  a  total  of  71  cases,  but  these  included  instances  of  COIN  campaigns  fought  “against  the  tide  of 
history” — for  example,  at  the  end  of  the  colonial  period  or  apartheid — and  one  case  of  indeterminate  outcome.  This  left  the 
59  core  cases  used  for  the  analyses  and  as  the  foundation  for  the  scorecard. 

2  Perfect  discrimination  of  59  historical  cases  is  by  no  means  a  guarantee  of  perfect  prediction  across  all  possible  cases.  Full 
details  of  the  historical  research  can  be  found  in  Christopher  Paul,  Colin  P.  Clarke,  Beth  Grill,  and  Molly  Dunigan,  Paths 
to  Victory:  Lessons  from  Modern  Insurgencies ,  Santa  Monica,  Calif.:  RAND  Corporation,  RR-291/1-OSD,  2013. 
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by  participants  that  many  of  the  positive  factors  identified  in  early  2013  Afghanistan  would 
be  difficult  for  the  Afghans  to  maintain  after  most  international  forces  are  withdrawn  in  2014. 

This  report  highlights  two  critical  areas  for  improvement.  These  factors  were  assessed 
as  absent  in  contemporary  Afghanistan  but  found  to  be  essential  to  success  in  historical 
COIN  campaigns:  (1)  disrupting  tangible  support  to  the  insurgents  and  (2)  a  demonstration 
(and  improvement)  of  commitment  and  motivation  on  the  part  of  the  Afghan  government  and 
Afghan  security  forces. 


Counterinsurgency  Scorecard:  Afghanistan  in  Early  2013  Relative 
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Findings  from  Previous  Research  on  Insurgency1 

The  RAND  report  Paths  to  Victory:  Lessons  from  Modern  Insurgencies  (RR-291/1-OSD)  used 
detailed  case  studies  of  the  71  insurgencies  begun  and  completed  worldwide  between  World 
War  II  (WWII)  and  2010  to  analyze  correlates  of  success  in  counterinsurgency  (COIN).2  That 
study  produced  nine  key  findings  regarding  success  in  COIN  in  modern  history: 

•  Seventeen  of  24  COIN  concepts  tested  receive  strong  support;  one  (“crush  them”)  has 
strong  evidence  against  it. 

•  Effective  COIN  practices  run  in  packs,  and  some  practices  are  always  in  the  pack:  tan¬ 
gible  support  reduction,  commitment  and  motivation,  and  flexibility  and  adaptability. 

•  Every  insurgency  is  unique,  but  not  so  much  that  it  matters  at  this  level  of  analysis;  the 
COIN  scorecard  discriminates  cases  into  wins  and  losses. 

•  Quality  is  more  important  than  quantity,  especially  where  paramilitaries  and  irregular 
forces  are  concerned. 

•  Governments  supported  by  external  actors  win  the  same  way  others  do. 

•  The  “iron  fist”  COIN  path,  focused  primarily  on  eliminating  the  insurgent  threat,  is  his¬ 
torically  less  successful. 

•  COIN  takes  time,  but  some  COIN  practices  help  end  insurgencies  sooner,  leading  to  a 
more  durable  postconflict  peace. 

•  COIN  takes  time:  After  good  COIN  practices  are  in  place,  the  average  insurgency  lasts 
roughly  six  more  years. 

•  Poor  beginnings  do  not  necessarily  lead  to  poor  ends. 

Of  particular  interest  is  the  second  part  of  the  third  finding,  “the  COIN  scorecard  dis¬ 
criminates  cases  into  wins  and  losses.”  The  COIN  scorecard  was  originally  developed  as  part 
of  an  earlier  study,  findings  from  which  were  published  in  the  2010  RAND  report  Victory 


1  Our  thanks  go  to  quality  assurance  reviewers  Joseph  Collins  and  John  Gordon,  who  provided  helpful  comments  on  a 
draft  of  this  report. 

2  Note  that  while  the  study  includes  71  detailed  case  studies,  only  59  of  these  cases  are  considered  “core”  cases  and 
informed  the  quantitative  and  comparative  analyses.  The  excluded  cases  are  those  that  were  “fought  against  the  tide  of 
history” — that  is,  cases  in  which  the  outcome  was  all  but  predetermined  by  exogenous  global  trends,  such  as  the  end 
of  colonialism  or  the  end  apartheid.  Many  of  these  “tide  of  history”  cases  are  individually  interesting  and  informative  but 
make  poor  comparative  cases  because,  often,  quite  well-designed  and  -executed  COIN  campaigns  ultimately  failed  because 
of  inexorable  changes  in  the  context  of  the  conflict.  Full  details  can  be  found  in  Christopher  Paul,  Colin  P.  Clarke,  Beth 
Grill,  and  Molly  Dunigan,  Paths  to  Victory:  Lessons  from  Modern  Insurgencies ,  Santa  Monica,  Calif.:  RAND  Corporation, 
RR-291/1-OSD,  2013. 
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Has  a  Thousand  Fathers:  Sources  of  Success  in  Counterinsurgency  (MG-964-OSD).  That  original 
COIN  scorecard  was  applied  to  Afghanistan  in  early  2011. 3  It  has  since  been  updated  based 
on  new  data  compiled  in  support  of  the  Paths  to  Victory  study.  Extending  that  work,  we  focus 
here  on  the  importance  of  maximizing  good  factors  and  minimizing  bad  factors  in  defeating 
insurgencies,  assessing  the  balance  of  these  factors  in  Afghanistan  in  early  2013. 


The  Counterinsurgency  Scorecard 

As  noted,  one  of  the  core  findings  of  the  previous  effort  was  that  a  historical  COIN  case’s  score 
on  a  scorecard  of  15  equally  weighted  good  factors  or  practices  minus  1 1  equally  weighted  bad 
factors  or  practices  perfectly  discriminated  case  outcomes  in  the  data.  Table  1  lists  these  good 
and  the  bad  factors. 

Across  the  59  core  cases  in  the  Paths  to  Victory  analyses,  taking  the  sum  of  the  good  minus 
the  bad  revealed  that  cases  with  a  good-minus-bad  score  of  +2  or  greater  were  always  won  by 
the  government,  and  cases  with  a  good-minus-bad  score  of — 1  or  lower  were  always  won  by  the 
insurgents.4  In  other  words,  scores  on  the  scorecard  perfectly  discriminated  the  historical  cases 
into  wins  and  losses.5  The  lesson  for  current  or  future  counterinsurgents  is  obvious:  Maximiz¬ 
ing  the  presence  of  good  factors  and  minimizing  the  presence  of  bad  factors  will  lead  to  the 
best  chances  for  success. 

Table  2  lists  the  59  core  insurgencies  from  Paths  to  Victory ,  along  with  the  dates  of  each 
conflict,  the  sum  of  good  factors  (of  a  possible  15),  the  sum  of  bad  factors  (of  a  possible  11), 
the  net  of  good-minus-bad  factors,  and  the  outcome  of  the  case  (either  a  government  loss  or  a 
government  win).6  Note  that  the  scorecard  score  does  indeed  discriminate  the  wins  from  the 
losses,  with  all  winning  COIN  forces  scoring  at  least  +2  and  all  losing  COIN  forces  scoring 
-1  or  lower. 


Scoring  Afghanistan  in  2013 

In  addition  to  providing  an  interesting  summary  of  historical  results  and  reflecting  historical 
correlates  of  success  in  COIN,  the  scorecard  is  meant  to  be  a  useful  prognostication  tool  for  cur¬ 
rent  and  future  COIN  efforts.  Although  we  do  not  assert  that  the  scorecard’s  ability  to  discrim¬ 
inate  wins  from  losses  perfectly  since  WWII  guarantees  its  ability  to  predict  the  outcomes  of 
ongoing  or  future  conflicts,  scores  for  these  COIN  efforts  should  offer  a  good  indication 


3  See  Christopher  Paul,  Counterinsurgency  Scorecard:  Afghanistan  in  Early  2011  Relative  to  the  Insurgencies  of  the  Past  30 
Years,  Santa  Monica,  Calif.:  RAND  Corporation,  OP-337-OSD,  2011.  For  background  on  the  scorecard’s  development,  see 
Christopher  Paul,  Colin  P.  Clarke,  and  Beth  Grill,  Victory  Has  a  Thousand  Fathers:  Sources  of  Success  in  Counterinsurgency, 
Santa  Monica,  Calif:  RAND  Corporation,  MG-964-OSD,  2010. 

4  Full  details  can  be  found  Paul,  Clarke,  Grill,  and  Dunigan,  2013. 

5  Note  that  perfect  discrimination  in  these  59  cases  does  not  guarantee  perfect  discrimination  across  all  possible  future 
cases.  With  a  larger  sample  size  (additional  cases  and  data),  it  is  possible — even  likely — that  the  estimated  model  would 
change  slightly.  In  fact,  this  occurred  when  the  data  set  went  from  30  cases  ( Victory  Has  a  Thousand  Fathers )  to  59  cases 
[Paths  to  Victory). 

6  For  full  details  on  case  selection  and  the  process  of  data  collection  and  factor  scoring  for  each  case,  see  Paul,  Clarke,  Grill, 
and  Dunigan,  2013.  Note  that  “loss”  also  includes  outcomes  assessed  as  “mixed,  favoring  insurgents,”  and  “win”  includes 
outcomes  assessed  as  “mixed,  favoring  COIN.” 
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Table  1 

Good  and  Bad  COIN  Factors  and  Practices 


15  Good  COIN  Practices 


11  Bad  COIN  Practices 


The  COIN  force  realized  at  least  two  strategic 
communication  factors. 

The  COIN  force  reduced  at  least  three  tangible  support 
factors. 

The  government  realized  at  least  one  government 
legitimacy  factor. 

Government  corruption  was  reduced/good  governance 
increased  since  the  onset  of  the  conflict. 

The  COIN  force  realized  at  least  one  intelligence  factor. 

The  COIN  force  was  of  sufficient  strength  to  force  the 
insurgents  to  fight  as  guerrillas. 

Unity  of  effort/unity  of  command  was  maintained. 

The  COIN  force  avoided  excessive  collateral  damage, 
disproportionate  use  of  force,  or  other  illegitimate 
application  of  force. 

The  COIN  force  sought  to  engage  and  establish  positive 
relations  with  the  population  in  the  area  of  conflict. 

Short-term  investments,  improvements  in  infrastructure  or 
development,  or  property  reform  occurred  in  the  area  of 
conflict  controlled  or  claimed  by  the  COIN  force. 

The  majority  of  the  population  in  the  area  of  conflict 
supported  or  favored  the  COIN  force. 

The  COIN  force  established  and  then  expanded  secure  areas. 

Government/COIN  reconstruction/development  sought/ 
achieved  improvements  that  were  substantially  above  the 
historical  baseline. 

The  COIN  force  provided  or  ensured  the  provision  of  basic 
services  in  areas  that  it  controlled  or  claimed  to  control. 

The  perception  of  security  was  created  or  maintained 
among  the  population  in  areas  that  the  COIN  force  claimed 
to  control. 


The  COIN  force  used  both  collective  punishment 
and  escalating  repression. 

There  was  corrupt  and  arbitrary  personalistic 
government  rule. 

Host-nation  elites  had  perverse  incentives  to 
continue  the  conflict. 

An  external  professional  military  engaged  in 
fighting  on  behalf  of  the  insurgents. 

The  host  nation  was  economically  dependent  on 
external  supporters. 

Fighting  was  initiated  primarily  by  the  insurgents. 

The  COIN  force  failed  to  adapt  to  changes  in 
adversary  strategy,  operations,  or  tactics. 

The  COIN  force  engaged  in  more  coercion  or 
intimidation  than  the  insurgents. 

The  insurgent  force  was  individually  superior  to 
the  COIN  force  by  being  either  more  professional 
or  better  motivated. 

The  COIN  force  or  its  allies  relied  on  looting  for 
sustainment. 

The  COIN  force  and  government  had  different 
goals  or  levels  of  commitment. 


of  whether  an  effort  is  on  a  path  likely  to  lead  to  success  or  on  one  that  is  less  likely  to  do  so. 
Specific  factors  scored  as  present  or  absent  should  also  provide  guidance  to  decisionmakers, 
strategists,  and  planners  about  areas  ripe  for  renewed  emphasis.7 

With  that  in  mind,  we  sought  to  complete  the  scorecard  for  ongoing  operations  in 
Afghanistan  as  of  early  2013.  At  the  time  of  this  writing,  COIN  efforts  in  Afghanistan  con¬ 
tinued  (and  continued  to  evolve)  against  a  backdrop  of  numerous  transitions,  with  interna¬ 
tional  forces  reducing  their  footprint  and  numbers  and  more  responsibility  being  transferred 
to  indigenous  institutions  and  forces.  Available  data  on  Afghanistan  are  often  lacking  in  reli- 


7  Note  that  the  scorecard  is  not  meant  to  be  a  mindless  checklist  or  substitute  for  a  thoughtful  and  considered  COIN 
strategy.  Which  factors  can  be  achieved,  how  difficult  they  are  to  achieve,  and  the  interrelationships  between  the  factors  all 
vary  with  the  specific  context,  and  every  insurgency  must  be  considered  within  its  own  context.  The  principal  value  of  the 
scorecard  is  for  diagnostic  purposes.  If  a  given  COIN  strategy  is  not  producing  a  positive  scorecard  balance,  careful  scrutiny 
of  the  scorecard  can  suggest  whether  the  strategy  is  on  the  right  track  (several  good  factors  are  likely  under  the  strategy  but 
have  not  yet  been  realized)  or  unlikely  to  lead  to  success  (unlikely  to  lead  to  many  good  factors,  especially  those  found  to  be 
essential  to  historical  COIN  wins). 
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Table  2 

59  Core  Cases  and  Scorecard  Scores 


Country  (Insurgency) 

Years 

Good  COIN 
Practices 

Bad  COIN 
Practices 

Total 

Score 

Outcome 

South  Vietnam 

1960-1975 

0 

-11 

-11 

COIN  loss 

Somalia 

1980-1991 

0 

-9 

-9 

COIN  loss 

Afghanistan  (post-Soviet) 

1992-1996 

0 

-9 

-9 

COIN  loss 

Kosovo 

1996-1999 

0 

-8 

-8 

COIN  loss 

Liberia 

1989-1997 

1 

-9 

-8 

COIN  loss 

Cambodia 

1967-1975 

0 

-7 

-7 

COIN  loss 

Moldova 

1990-1992 

1 

-8 

-7 

COIN  loss 

Georgia/Abkhazia 

1992-1994 

0 

-7 

-7 

COIN  loss 

Zaire  (anti-Mobutu) 

1996-1997 

0 

-7 

-7 

COIN  loss 

Nicaragua  (Somoza) 

1978-1979 

0 

-6 

-6 

COIN  loss 

Chechnya  1 

1994-1996 

1 

-7 

-6 

COIN  loss 

Bosnia 

1992-1995 

0 

-6 

-6 

COIN  loss 

Laos 

1959-1975 

2 

-7 

-5 

COIN  loss 

Nagorno-Karabakh 

1992-1994 

0 

-5 

-5 

COIN  loss 

Democratic  Republic  of  the  Congo 
(anti-Kabila) 

1998-2003 

1 

-5 

-4 

COIN  loss 

Rwanda 

1990-1994 

2 

-6 

-4 

COIN  loss 

Bangladesh 

1971 

2 

-6 

-4 

COIN  loss 

Afghanistan  (Taliban) 

1996-2001 

2 

-6 

-4 

COIN  loss 

Kampuchea 

1978-1992 

0 

-3 

-3 

COIN  loss 

Cuba 

1956-1959 

3 

-6 

-3 

COIN  loss 

Eritrea 

1961-1991 

1 

-4 

-3 

COIN  loss 

Sudan  (Sudanese  People's  Liberation  Army 
[SPLA]) 

1984-2004 

1 

-4 

-3 

COIN  loss 

Afghanistan  (anti-Soviet) 

1978-1992 

2 

-5 

-3 

COIN  loss 

Burundi 

1993-2003 

1 

-3 

-2 

COIN  loss 

Yemen 

1962-1970 

1 

-3 

-2 

COIN  loss 

Lebanese  Civil  War 

1975-1990 

5 

-7 

-2 

COIN  loss 

Tajikistan 

1992-1997 

2 

-4 

-2 

COIN  loss 

Nepal 

1997-2006 

3 

-4 

-1 

COIN  loss 

Indonesia  (East  Timor) 

1975-2000 

3 

-4 

-1 

COIN  loss 

Nicaragua  (Contras) 

1981-1990 

3 

-4 

-1 

COIN  loss 

Papua  New  Guinea 

1988-1998 

2 

-3 

-1 

COIN  loss 

Iraqi  Kurdistan 

1961-1975 

4 

-2 

2 

COIN  win 

Western  Sahara 

1975-1991 

4 

-2 

2 

COIN  win 

Argentina 

1969-1979 

5 

-2 

3 

COIN  win 

Oman  (Imamate  Uprising) 

1957-1959 

4 

-1 

3 

COIN  win 

Croatia 

1992-1995 

5 

-2 

3 

COIN  win 

Guatemala 

1960-1996 

8 

-4 

4 

COIN  win 

Tibet 

1956-1974 

7 

-3 

4 

COIN  win 

Sri  Lanka 

1976-2009 

6 

-1 

5 

COIN  win 

Mozambique  (Mozambican  National 

1976-1995 

8 

-3 

5 

COIN  win 

Resistance  [RENAMO]) 
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Table  2 — Continued 


Country  (Insurgency) 

Years 

Good  COIN 
Practices 

Bad  COIN 
Practices 

Total 

Score 

Outcome 

Turkey  (Kurdistan  Workers'  Party  [PKK]) 

1984-1999 

8 

-2 

6 

COIN  win 

Indonesia  (Aceh) 

1976-2005 

8 

-2 

6 

COIN  win 

Algeria  (Armed  Islamic  Group  [G 1  A]) 

1992-2004 

6 

0 

6 

COIN  win 

Baluchistan 

1973-1978 

9 

-2 

7 

COIN  win 

Uganda  (Allied  Democratic  Forces  [ADF]) 

1986-2000 

7 

0 

7 

COIN  win 

Northern  Ireland 

1969-1999 

9 

-1 

8 

COIN  win 

Jordan 

1970-1971 

9 

0 

9 

COIN  win 

Indonesia  (Darul  Islam) 

1958-1962 

10 

0 

10 

COIN  win 

Angola  (National  Union  for  the  Total 
Independence  of  Angola  [UNITA]) 

1975-2002 

12 

-2 

10 

COIN  win 

Greece 

1945-1949 

12 

-2 

10 

COIN  win 

Uruguay 

1963-1972 

10 

0 

10 

COIN  win 

Malaya 

1948-1955 

13 

-2 

11 

COIN  win 

El  Salvador 

1979-1992 

12 

-1 

11 

COIN  win 

Oman  (Dhofar  Rebellion) 

1965-1975 

13 

-1 

12 

COIN  win 

Peru 

1980-1992 

14 

-2 

12 

COIN  win 

Sierra  Leone 

1991-2002 

14 

-1 

13 

COIN  win 

Senegal 

1982-2002 

13 

0 

13 

COIN  win 

Philippines  (Moro  National  Liberation  Front 
[MNLF]) 

1971-1996 

14 

-1 

13 

COIN  win 

Philippines  (Huk  Rebellion) 

1946-1956 

15 

0 

15 

COIN  win 

ability  and  validity  due  to  inconsistent  collection  practices,  inconsistent  reporting,  and  genuine 
ambiguity  in  many  Afghan  subcontexts.  Because  it  can  be  difficult  and  contentious  to  identify 
specific  scorecard  factors  as  present  or  absent,  we  conducted  an  expert  elicitation  exercise  with 
a  panel  of  19  subject-matter  experts  who  were  knowledgeable  about  ongoing  operations  in 
Afghanistan,  keeping  responses  and  participants  anonymous.  Panelists  included  RAND  staff 
with  expertise  on  Afghanistan  or  who  had  deployed  to  Afghanistan;  serving  field-grade  U.S. 
military  officers  with  multiple  (and  recent)  deployments  to  Afghanistan;  military  veterans  who 
were  current  on  COIN  research;  civilian  defense  representatives  from  the  Office  of  the  Secre¬ 
tary  of  Defense,  the  Joint  Staff,  and  U.S.  Central  Command;  government  civilians  with  expe¬ 
rience  in  Afghanistan;  staff  from  the  U.S.  Agency  for  International  Development;  university 
faculty  members;  journalists;  and  experts  from  other  prominent  think  tanks.  Full  details  of  the 
expert  elicitation  process  are  provided  in  the  appendix. 

In  keeping  with  the  approach  used  in  the  previous  studies  to  quantify  the  historical 
cases  and  to  create  and  update  the  scorecard — and  consistent  with  the  prescribed  use  of  the 
scorecard — participants  were  asked  to  make  worst-case  assessments.  To  be  considered  present, 
positive  factors  were  required  to  be  present  over  the  preponderance  of  the  area  of  conflict,  not 
just  in  isolated  locations.  Similarly,  bad  factors  were  counted  as  present  if  they  occurred  with 
any  frequency  greater  than  isolated  incidents.  So,  good  factors  needed  to  be  preponderantly 
present,  but  bad  factors  needed  only  to  be  routinely  present  in  certain  areas  or  among  certain 
segments  of  the  force.  (Further  details  about  the  scoring  instructions  can  be  found  in  the 
appendix.) 
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The  expert  elicitation  provided  scores  for  the  42  specific  factors  or  subfactors  in  the  score- 
card  necessary  to  calculate  the  15  good  and  11  bad  factors  and  practices  listed  in  Table  l.s 
Tables  3  and  4  show  all  42  factors  or  subfactors  and  how  they  roll  up  into  the  15  good  and 
1 1  bad  factors,  respectively.  Where  there  was  consensus  on  the  presence  or  absence  of  a  factor, 
it  was  scored  “1”  if  present  or  “0”  if  absent;  where  there  was  not  consensus  among  the  panel¬ 
ists,  factors  were  scored  “0.5”  for  neither  present  nor  absent.8 9  The  detailed  scorecard  and  the 
consensus  assessment  of  each  factor  and  subfactor  are  presented  in  Tables  3  and  4. 

The  scorecard  as  completed  through  expert  elicitation  indicates  that  eight  good  scorecard 
factors  and  six  bad  factors  were  present  in  Afghanistan  in  early  2013.  The  eight  good  factors 
include  four  half-points  and  thus  represent  a  range  from  6  to  10.  The  six  bad  factors  include 
two  half-points,  thus  representing  a  range  between  4  and  8.  Subtracting  6  from  8  provides  a 
scorecard  net  result  of  +2.  (With  all  the  half-points  maximized  in  one  direction  or  the  other, 
this  represents  a  range  of -2  to  +6,  a  band  of  uncertainty  that  can  be  interpreted  as  something 
similar  to  a  confidence  interval.) 

Table  3 

Good  Factors  Present  in  Early  2013  Afghanistan  (Total:  8  of  15) 


Good  Factors 


1.  COIN  force  realizes  at  three  two  strategic  communication  factors  (Score  1  if  sum  of 
a  through  e  is  at  least  2) 

a.  COIN  force  and  government  actions  consistent  with  messages  (delivering  on  promises) 
(Score  1  if  YES) 

b.  COIN  force  maintains  credibility  with  populations  in  the  area  of  conflict  (includes 
expectation  management)  (Score  1  if  YES) 

c.  Messages/themes  coherent  with  overall  COIN  approach  (Score  1  if  YES) 


d.  COIN  force  avoids  creating  unattainable  expectations  (Score  1  if  YES) 


e.  Themes  and  messages  coordinated  for  all  involved  government  agencies  (Score  1  if  YES) 

2.  COIN  force  reduces  at  least  three  tangible  support  factors  (Score  1  if  sum  of  a  through  j  is 
at  least  3) 

a.  Flow  of  cross-border  insurgent  support  significantly  decreased,  or  remains  dramatically 
reduced,  or  largely  absent  (Score  1  if  YES) 

b.  Important  external  support  to  insurgents  significantly  reduced  (Score  1  if  YES) 


c.  Important  internal  support  to  insurgents  significantly  reduced  (Score  1  if  YES) 


d.  Insurgents'  ability  to  replenish  resources  significantly  diminished  (Score  1  if  YES) 


8  Note  that  the  expert  elicitation  considered  42  factors  rather  than  15  +  11  =  26  factors  because  some  are  summary  factors 
(such  as  “COIN  force  realizes  at  least  two  strategic  communication  factors”)  that  rely  on  multiple  subordinate  factors  for 
calculation. 

9  Consensus,  for  this  exercise,  was  considered  to  be  agreement  among  at  least  70  percent  of  panelists.  In  practice,  for  most 
consensus  factors,  the  degree  of  accord  was  considerably  higher.  The  raw  averages  (equivalent  to  the  percentage  of  panelists 
in  agreement)  are  listed  in  Table  A.l  in  the  appendix. 
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Table  3 — Continued 


Good  Factors 


A  B 


e.  Insurgents  unable  to  maintain  or  grow  force  size  (Score  1  if  YES) 

f.  COIN  force  efforts  resulting  in  increased  costs  for  insurgent  processes  (Score  1  if  YES) 

g.  COIN  forces  effectively  disrupt  insurgent  recruiting  (Score  1  if  YES) 

h.  COIN  forces  effectively  disrupt  insurgent  materiel  acquisition  (Score  1  if  YES) 

i.  COIN  forces  effectively  disrupt  insurgent  intelligence  (Score  1  if  YES) 


j.  COIN  forces  effectively  disrupt  insurgent  financing  (Score  1  if  YES) 

3.  Government  realizes  at  least  one  government  legitimacy  factors  (Score  1  if  sum  of  a  and  b 
is  at  least  1) 

b.  Government  leaders  selected  in  a  manner  considered  just  and  fair  by  majority  of 
population  in  the  area  of  conflict  (Score  1  if  YES) 

c.  Majority  of  citizens  in  the  area  of  conflict  view  government  as  legitimate  (Score  1  if  YES) 

4.  Government  corruption  reduced/good  governance  increased  since  onset  of  conflict 
(Score  1  if  YES) 

5.  COIN  force  realizes  at  least  one  intelligence  factor  (Score  1  if  sum  of  a  and  b  is  at  least  1) 

a.  Intelligence  adequate  to  support  kill/capture  or  engagements  on  COIN  force's  terms 
(Score  1  if  YES) 

b.  Intelligence  adequate  to  allow  COIN  force  to  disrupt  insurgent  processes  or  operations 
(Score  1  if  YES) 

6.  COIN  force  of  sufficient  strength  to  force  insurgents  to  fight  as  guerrillas  (Score  1  if  YES) 


7.  Unity  of  effort/unity  of  command  maintained  (Score  1  if  YES) 

8.  COIN  force  avoids  excessive  collateral  damage,  disproportionate  use  of  force,  or  other 
illegitimate  applications  of  force  (Score  1  if  YES) 

9.  COIN  force  seeks  to  engage  and  establish  positive  relations  with  population  in  area  of  conflict 
(Score  1  if  YES) 

10.  Short-term  investments,  improvements  in  infrastructure/development,  or  property  reform  in 
area  of  conflict  controlled  or  claimed  by  COIN  force  (Score  1  if  YES) 

11.  Majority  of  population  in  area  of  conflict  supports/favors  COIN  forces  (Score  1  if  YES) 


12.  COIN  force  establishes  and  then  expands  secure  areas  (Score  1  if  YES) 

13.  Government/COIN  reconstruction/development  sought/achieved  improvements  substantially 
above  historical  baseline  (Score  1  if  YES) 

14.  COIN  force  provides  or  ensures  provision  of  basic  services  in  areas  it  controls  or  claims  to 
control  (Score  1  if  YES) 

15.  Perception  of  security  created  or  maintained  among  population  in  areas  COIN  force  claims  to 
control  (Score  1  if  YES) 
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Table  4 

Bad  Factors  Present  in  Early  2013  Afghanistan  (Total:  6  of  11) 


Bad  Factors 


1.  COIN  force  uses  both  collective  punishment  and  escalating  repression 
(Score  1  if  sum  of  a  and  b  is  at  least  1) 

a.  COIN  force  employs  escalating  repression  (Score  1  if  YES) 

b.  COIN  force  employs  collective  punishment  (Score  1  if  YES) 

2.  Government  involves  corrupt  and  arbitrary  personalistic  rule  (Score  1  if  YES) 

3.  Country  elites  have  perverse  incentives  to  continue  conflict  (Score  1  if  YES) 

4.  External  professional  military  engaged  in  fighting  on  behalf  of  insurgents  (Score  1  if  YES) 

5.  Host  nation  is  economically  dependent  on  external  supporters  (Score  1  if  YES) 

6.  Fighting  primarily  initiated  by  the  insurgents  (Score  1  if  YES) 

7.  COIN  force  fails  to  adapt  to  changes  in  adversary  strategy,  operations,  or  tactics 
(Score  1  if  YES) 

8.  COIN  force  engages  in  more  coercion/intimidation  than  insurgents  (Score  1  if  YES) 

9.  Insurgent  force  individually  superior  to  COIN  force  by  being  either  more  professional  or  better 
motivated  (Score  1  if  YES) 

10.  COIN  force  or  allies  rely  on  looting  for  sustainment  (Score  1  if  YES) 

11.  COIN  force  and  government  have  different  goals/levels  of  commitment  (Score  1  if  YES) 


Table  5  shows  where  these  scores  (8  good,  6  bad,  +2  net)  would  fit  in  among  the  59  core 
insurgencies  from  Paths  to  Victory.  Table  5  lists  a  subset  of  the  cases  in  Table  2;  it  includes  only 
the  cases  with  similar  good,  bad,  and  net  scores  to  those  found  for  early  2013  Afghanistan. 
(See  Table  2  for  the  full  list  of  59  core  cases.)  An  overall  scorecard  score  of  +2  is  tied  with  the 
lowest  historical  score  for  a  case  won  by  a  COIN  force.  Having  eight  good  factors  puts  early 
2013  Afghanistan  in  the  middle  of  the  pack  of  counterinsurgent  winners,  several  points  above 
the  floor  for  COIN  wins  (four).  Even  accounting  for  uncertainty  due  to  half-points,  the  lower 
bound  for  the  good  factor  estimate  (six)  is  consonant  with  all  historical  COIN  wins.  Having 
six  bad  factors,  however,  puts  early  2013  Afghanistan  two  points  below  the  record  of  the  worst¬ 
scoring  historical  COIN  winner  (Guatemala,  at  -4)  and  is  much  more  typical  of  scores  for 
cases  in  which  governments  lost.  (Four  bad  factors  is  the  most  optimistic  bound  for  the  Delphi 
estimate  for  contemporary  Afghanistan,  discounting  half-points.) 

Without  taking  into  account  specific  factors  or  other  findings  from  the  broader  research, 
the  scorecard  results  are  sufficient  for  cautious  optimism,  particularly  because  the  scoring 
for  the  exercise  was  based  on  worst-case  assessments.  The  number  of  good  factors  present  is 
strong;  even  considering  the  lower  bound  of  the  uncertainty  band  due  to  nonconsensus  half¬ 
points  (a  range  of  6  to  10),  only  historical  winners  had  at  least  six  good  factors.  The  total  score 
(good  minus  bad),  +2,  is  above  the  historical  threshold  for  a  COIN  win  and  is  tied  with  the 
lowest  historical  score.  The  overall  band  of  uncertainty  (-2  to  +6)  does  straddle  the  lowest- 
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Table  5 

Relevant  Subset  of  Cases  and  Scores  with  Early  2013  Afghanistan  Scores  Overlaid 


Country  (Insurgency) 


Years 


Nicaragua  (Somoza)  1978-1979 

Chechnya  I  1994-1996 

Bosnia  1992-1995 

Laos  1959-1975 

Nagorno-Karabakh  1992-1994 

Democratic  Republic  of  the  Congo  (anti-  1998-2003 

Kabila) 

Rwanda  1990-1994 

Bangladesh  1971 

Afghanistan  (Taliban)  1996-2001 

Kampuchea  1978-1992 

Cuba  1956-1959 

Eritrea  1961-1991 

Sudan  (SPLA)  1984-2004 

Afghanistan  (anti-Soviet)  1978-1992 

Burundi  1993-2003 

Yemen  1962-1970 

Lebanese  Civil  War  1975-1990 

Tajikistan  1992-1997 

Nepal  1997-2006 

Indonesia  (East  Timor)  1975-2000 

Nicaragua  (Contras)  1981-1990 

Papua  New  Guinea  1988-1998 

Iraqi  Kurdistan  1961-1975 

Western  Sahara  1975-1991 

Argentina  1968-1979 

Oman  (Imamate  Uprising)  1957-1959 

Croatia  1992-1995 

Guatemala  1960-1996 

Tibet  1956-1974 

Sri  Lanka  1976-2009 

Mozambique  (RENAMO)  1976-1995 

Turkey  (PKK)  1984-1999 

Indonesia  (Aceh)  1976-2005 


Good  COIN 
Practices 


Bad  COIN 
Practices 


-7 


-7 

-5 

-5 


Total 

Score 


-6 

-6 

-6 

-5 

-5 

-4 

-4 

-4 

-4 

-3 

-3 

-3 

-3 

-3 

-2 

-2 

-2 

-2 

-1 

-1 

-1 

-1 


Outcome 


COIN  loss 
COIN  loss 
COIN  loss 
COIN  loss 
COIN  loss 
COIN  loss 

COIN  loss 
COIN  loss 
COIN  loss 
COIN  loss 
COIN  loss 
COIN  loss 
COIN  loss 
COIN  loss 
COIN  loss 
COIN  loss 
COIN  loss 
COIN  loss 
COIN  loss 
COIN  loss 
COIN  loss 
COIN  loss 
COIN  win 
COIN  win 
COIN  win 
COIN  win 
COIN  win 
COIN  win 
COIN  win 
COIN  win 
COIN  win 
COIN  win 
COIN  win 


NOTE:  Shading  indicates  scores  equivalent  to  Afghanistan's  in  each  category. 


scoring  historical  winners  and  the  highest-scoring  historical  losses.  The  fact  that  the  sum  of 
bad  factors  (six)  is  greater  than  the  sum  of  bad  factors  for  any  historical  winners  is  a  concern, 
however.  From  a  policy  perspective,  these  findings  recommend  seeking  further  improvements  to 
increase  the  overall  scorecard  score,  adding  critical  missing  good  factors  but  with  a  particular  focus 
on  diminishing  the  number  bad  factors  to  increase  confidence  and  the  likelihood  of  COIN  success 
in  Afghanistan. 
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Of  course,  the  findings  of  the  previous  RAND  studies  make  this  a  general  recommen¬ 
dation;  COIN  forces  should  always  seek  to  maximize  good  factors  and  minimize  bad  factors. 
The  scorecard  and  elicitation  exercise  offer  additional  value  in  identifying  the  specific  factors 
that  are  present  or  absent.  The  next  section  discusses  these  factors  and  provides  suggestions 
regarding  which  factors  might  be  improved  upon.  By  way  of  preview,  critical  factors  that  are 
absent  include  a  demonstration  of  commitment  and  motivation  on  the  part  of  the  Afghan  gov¬ 
ernment  and  Afghan  security  forces  and  the  interdiction  of  tangible  support  to  the  insurgents 
(interrupting  flows  of  materiel  and  other  forms  of  support  both  from  within  Afghanistan  and 
across  borders). 

Detailed  Factors  in  the  Current  Case 

Tables  3  and  4  present  all  factors  and  subfactors  in  the  scorecard  and  the  factors  scored  as  pres¬ 
ent  or  absent  during  the  expert  elicitation  exercise.  In  each  row,  a  “1”  indicates  that  the  consen¬ 
sus  view  was  that  the  factor  or  practice  was  deemed  to  be  present  in  the  current  conflict;  a  “0” 
indicates  that  the  consensus  view  was  that  the  factor  was  absent.10  When  reviewing  the  scores, 
remember  that  participants  were  asked  to  make  worst-case  assessments;  isolated  anecdotes  or 
limited  accounts  of  success  in  these  areas  were  not  considered  sufficient  grounds  to  score  a 
factor  (be  it  a  bad  factor  or  a  good  factor)  as  present.  Where  the  score  is  “0.5,”  there  was  no  clear 
consensus  among  participants.  Consensus  was  reached  for  34  of  the  42  factors  scored  directly 
by  the  panel.  These  factors  are  discussed  in  greater  detail  later.  Rows  beginning  with  numbers 
are  primary  factors  (i.e.,  the  top-level  factors  listed  Table  1);  rows  beginning  with  lowercase  let¬ 
ters  are  the  subfactors  that  constitute  the  primary  factors.  (All  subfactors  were  directly  scored 
by  panelists,  as  were  any  primary  factors  without  subfactors.) 

Critical  Factors 

The  discussion  and  findings  from  the  previous  research  suggest  that  some  of  the  good  factors 
that  the  participants  identified  as  absent  are  particularly  concerning. 

Early  2013  Afghanistan  was  scored  as  having  eight  of  a  possible  15  good  primary  fac¬ 
tors  present  and,  thus,  seven  (actually,  five  and  four  half-points)  absent.  The  absent  factors 
and  those  for  which  there  was  not  consensus  agreement  of  presence  or  absence  are  as  follows 
(with  those  for  which  there  was  not  consensus  denoted  with  a  0.5  in  parentheses): 

•  COIN  force  realizes  at  least  two  strategic  communication  factors. 

•  COIN  force  reduces  at  least  three  tangible  support  factors. 

•  Government  realizes  at  least  two  government  legitimacy  factors. 

•  Government  corruption  reduced/good  governance  increased  since  onset  of  conflict. 

•  Unity  of  effort/ unity  of  command  maintained. 

•  COIN  force  established  and  then  expanded  secure  areas  (0.5). 

•  Government/COIN  reconstruction/development  sought/achieved  improvements  sub¬ 
stantially  above  historical  baseline  (0.5). 


10  In  all  cases,  the  threshold  for  consensus  was  70-percent  agreement  among  the  expert  participants.  The  percentages  of 
respondents  indicating  each  factor  as  present  or  absent  can  be  found  in  Table  A.l  in  the  appendix. 
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•  COIN  force  provides  or  ensures  provision  of  basic  services  in  areas  it  controls  or  claims 
to  control  (0.5). 

•  Perception  of  security  created  or  maintained  among  population  in  areas  COIN  force 
claims  to  control  (0.5). 

These  absences  suggest  possible  areas  for  improvement,  but  one  is  particularly  critical 
based  on  the  historical  results:  the  interdiction  of  tangible  support  to  the  insurgents.  This 
concerns  the  extent  to  which  COIN  forces  have  cut  flows  of  support  (e.g.,  materiel,  person¬ 
nel,  financing,  intelligence)  to  the  insurgents.  The  expert  panel  scored  only  one  of  the  ten 
tangible  support  reduction  subfactors  as  present;  for  the  parent  factor  to  be  scored  “1”  on 
the  scorecard,  at  least  three  subfactors  must  be  present  (and,  obviously,  even  more  would  be 
better).  This  is  particularly  concerning  because,  in  the  59  core  historical  cases,  all  the  COIN 
winners  disrupted  at  least  three  tangible  support  factors  and  none  of  the  losers  did.  The  discus¬ 
sion  revealed  that  the  insurgents  in  Afghanistan  meet  their  tangible  support  needs  from  many 
sources,  none  of  which  has  been  significantly  disrupted:  supporters  across  the  border  in  Paki¬ 
stan  (and  in  other  countries),  profits  from  drug  trafficking  and  other  criminal  activities,  and 
supportive  populations  (sometimes  coerced)  inside  Afghanistan.  Disrupting  tangible  support 
would  require  significant  efforts  to  interrupt  support  flows  from  all  of  these  sources,  but  this 
may  well  be  necessary  for  COIN  forces  to  prevail. 

Along  with  the  absence  of  certain  critical  good  factors,  the  presence  of  several  critical  bad 
factors  suggests  possible  areas  for  improvement.  Five  (and  two  half-points)  from  the  11  bad  fac¬ 
tors  were  scored  as  present  by  the  expert  panel: 

•  Government  involves  corrupt  and  arbitrary  personalistic  rule. 

•  Country  elites  have  perverse  incentives  to  continue  conflict. 

•  Host  nation  is  economically  dependent  on  external  supporters. 

•  Fighting  primarily  initiated  by  the  insurgents. 

•  Insurgent  force  individually  superior  to  the  COIN  force  by  being  either  more  professional 
or  better  motivated  (0.5). 

•  COIN  force  or  allies  rely  on  looting  for  sustainment  (0.5). 

•  COIN  force  and  government  have  different  goals/level  of  commitment. 

Not  only  is  the  total  number  of  bad  factors  present  concerning,  but  so  are  the  factors 
themselves.  Four  of  the  five  bad  practices  present  and  the  two  nonconsensus  half-points  are 
all  in  a  single  category  from  the  previous  research:  commitment  and  motivation.  The  extent 
to  which  the  government  and  COIN  forces  facing  the  insurgency  demonstrate  their  resolve  to 
defeat  that  insurgency  is  one  of  the  strongest  predictors  of  success  in  the  historical  cases.  In  the 
historical  cases,  no  government  with  four  or  more  of  eight  factors  indicating  low  commitment 
and  motivation  managed  to  prevail.  The  expert  panel  agreed  that  Afghanistan  has  too  many 
of  these  factors  present.11  This  suggests  that  the  Afghan  insurgency  will  be  defeated  only  if  the 
Afghan  government  and  security  forces  are  able  to  establish  and  demonstrate  their  commit¬ 
ment  to  doing  so. 


11  Of  the  eight  commitment  and  motivation  factors  referenced,  only  six  are  included  in  the  scorecard.  The  expert  panel 
agreed  that  four  of  those  six  were  present,  and  consensus  was  not  reached  on  two  others  (half-points). 
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Notes  on  Factors  for  Which  Consensus  Was  Lacking 

Although  the  participants  in  the  expert  elicitation  had  or  achieved  consensus  on  most  factors 
(34  of  42),  some  factors  remained  sources  of  disagreement  throughout  the  exercise.  Several  of 
them  merit  specific  mention. 

One  such  contentious  factor  was  “Intelligence  adequate  to  support  kill/capture  or  engage¬ 
ments  with  insurgents  on  COIN  force’s  terms.”  There  were  several  layers  of  disagreement.  First, 
there  were  disagreements  about  the  quality  of  intelligence  available  and  to  whom.  A  contrast 
was  drawn  between  the  kind  of  intelligence  that  drives  raids  by  special  operations  forces  (SOF), 
which  is  certainly  sufficient  for  those  missions,  and  the  kind  of  intelligence  available  for  con¬ 
ventional  formations.  As  one  panelist  contended,  “The  average  conventional  unit  (at  the  tacti¬ 
cal  level — battalion  or  brigade)  conducts  patrols  and  operations  that  amount  to  movement  to 
contact,  as  opposed  to  deliberate  operations  against  a  targeted  or  identified  insurgent.  Action¬ 
able  intelligence  is  usually  only  present  at  the  strategic  level  or,  at  least,  for  strategic-level  SOF 
assets.”  Others  disagreed.  A  second  point  of  contention  surrounded  what  should  be  considered 
“adequate.”  For  example,  while  many  panelists  accepted  that  intelligence  was  indeed  sufficient 
to  fuel  targeted  raids  by  SOF,  one  panelist  asserted  that  this  was  not  the  case,  asking,  “Where  is 
Mullah  Omar?  Haqqani?  Flekmatyar?  Even  second-tier  leaders  remain  at  large.”  Finally,  some 
panelists  questioned  the  Afghans’  ability  to  perform  the  intelligence  function  that  has,  to  date, 
been  overseen  by  coalition  forces,  contending  that  whatever  the  characterization  of  current 
intelligence  effectiveness,  it  would  decline  considerably  as  coalition  forces  withdraw. 

Another  disputed  factor  was  “COIN  force  establishes  and  then  expands  secure  areas.” 
Panelists  agreed  that  COIN  forces  had  done  this  and  had  historically  done  so  successfully 
during  the  conflict,  but  they  were  split  on  whether  it  was  still  taking  place  in  early  2013.  While 
some  noted  recent  reports  contending  continued  progress  in  extending  government  territo¬ 
rial  control,  others  disagreed.  One  panelist  argued,  “The  retrograde  affects  this  significantly: 
Secure  areas  are  actually  shrinking  since  ISAF  [International  Security  Assistance  Force]  is  pull¬ 
ing  back  to  major  bases,  and  ANSF  [Afghan  National  Security  Forces]  are  focusing  on  metro¬ 
politan  areas  and  the  Ring  Road.”  Although  the  panel  could  not  agree  on  whether  secure  areas 
were  growing  or  shrinking,  the  discussion  revealed  a  broadly  shared  concern  about  the  ability 
of  Afghan  security  forces  to  do  this  on  their  own.  Summarized  one  panelist,  “The  expansion  of 
secure  areas  has  required  enormous  international  resources,  which  will  be  difficult  to  sustain 
as  international  forces  roll  up.  ANSF  will  have  to  take  a  different  approach  to  the  insurgency, 
which,  in  effect,  will  concede  large  areas  that  are  already  de  facto  under  Taliban  control.” 

Related  was  disagreement  on  “Perception  of  security  created  or  maintained  among  popu¬ 
lation  in  areas  COIN  force  claims  to  control.”  Several  panelists  reported  progress  on  this  factor, 
noting  surveys  indicating  that  perceptions  of  security  were  “trending  positive.”  Others  pointed 
out  significant  regional  differences  in  the  perception  of  security,  accepting  that  there  is  a  strong 
perception  of  security  in  some  parts  of  the  country,  but  “in  contested  areas,  not  so  much.” 
Panelists  could  not  agree  on  the  scope  of  perceptions  of  security  (such  as  where  in  the  country 
such  perceptions  were  present  and  where  they  were  absent),  nor  on  what  level  of  perception 
was  required,  with  some  panelists  contending  that  a  lower-threshold  (“Afghan  good-enough”) 
perception  of  security  had  been  broadly  established. 

“COIN  force  provides  or  ensures  provision  of  basic  services  in  areas  it  controls  or  claims 
to  control”  was  another  factor  on  which  the  panel  could  not  reach  consensus.  Under  dispute 
were  both  the  definition  of  a  basic  service  and  the  extent  to  which  basic  services  were  pro¬ 
vided.  Some  panelists  were  content  to  leave  “basic  services”  unspecified,  some  defined  services 
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by  their  absence  (e.g.  “water,  electricity,  and  dispute  resolution”),  and  some  listed  them  more 
exhaustively  (health,  education,  vocational  training,  potable  water/sanitation,  energy,  infra¬ 
structure/roads,  irrigation,  and  social  protection).  There  was  agreement  that  most  basic  services 
were  being  provided  in  major  cities,  but  there  was  disagreement  about  the  extent  of  service 
provision  outside  the  cities  and  the  adequacy  of  the  services  provided.  This  was  another  factor 
for  which  panelists — whether  opining  that  services  were  being  broadly  provided  or  not — 
anticipated  future  deterioration  in  provision  with  decreased  international  support  and 
participation. 

Interestingly,  the  panel  disputed  the  factor  “Insurgent  force  individually  superior  to 
COIN  force  by  being  either  more  professional  or  better  motivated,”  one  of  the  1 1  bad  factors. 
The  panelists  did  not  disagree  about  the  relative  superiority  of  NATO  and  U.S.  troops  over  the 
insurgents,  but  there  was  some  disagreement  about  the  relative  merits  of  the  insurgents  and 
the  Afghan  security  forces.  While  there  was  broad  agreement  that  most  of  the  Afghan  security 
forces  (particularly  the  army  and  the  national  police)  were  better  trained  and  equipped  than 
the  insurgents  (and  thus  more  professional),  views  on  the  strength  of  their  motivation  dif¬ 
fered.  Several  panelists  asserted  that  the  insurgents  were,  on  average,  more  motivated  than  the 
Afghan  security  forces  and  that  made  them,  on  average,  individually  superior.  The  majority 
disagreed,  but  the  minority  was  sufficiently  large  to  preclude  consensus. 

The  final  contentious  factor  was  also  from  among  the  11  bad  factors,  “COIN  force  or 
allies  rely  on  looting  for  sustainment.”  None  of  the  participants  asserted  that  members  of 
ISAF  were  engaged  in  looting,  but  many  asserted  that  there  has  been  looting,  graft,  theft,  and 
bribery  on  the  part  of  various  Afghan  security  forces.  The  dispute,  then,  involved  two  issues: 
whether  this  behavior  was  sufficiently  widespread  to  count  as  present  and  whether  the  looting 
was  “for  sustainment.”  Consensus  was  not  reached. 

It  is  interesting  to  note  several  common  themes  that  were  prevalent  in  the  discussion  and 
bridged  numerous  factors  and  topics,  despite  disagreement  on  the  scoring  of  the  factors  them¬ 
selves.  One  such  theme  was  the  uncertainty  regarding  the  long-term  situation  in  the  country 
following  U.S.  withdrawal.  That  this  uncertainty  was  mentioned  extensively  in  the  panelists’ 
comments  indicates  potential  uncertainty  in  the  relevance  of  current  scores  to  the  future  situ¬ 
ation  in  Afghanistan.  This  issue  is  discussed  further  in  the  next  section. 

A  second  theme  was  the  distinction  between  various  COIN  activities  in  rural  areas  versus 
population  centers  and  a  distinction  in  the  degree  to  which  the  population  in  each  area  is 
receptive  to  COIN  forces.  Because  participants  were  asked  to  make  worst-case  assessments 
across  the  entire  country  in  scoring  each  factor  (with  good  factors  having  to  be  predominantly 
present  and  bad  factors  having  only  to  be  routinely  present  to  be  scored  as  such),  we  assume 
for  the  sake  of  the  scorecard  analysis  that  these  regional  distinctions  are  irrelevant.  However, 
the  extent  to  which  this  issue  was  raised  in  the  discussion  indicates  that  it  may  have  particular 
bearing  in  the  context  of  the  conflict  in  Afghanistan  and  should  be  considered  when  look¬ 
ing  for  ways  to  improve  COIN  efforts  there.  This  is  particularly  so  with  regard  to  some  of  the 
scorecard’s  more  population-focused  factors,  such  as  creating  or  maintaining  the  perception 
of  security  among  the  population  and  the  COIN  force’s  provision  of  basic  services  in  areas  it 
control  or  claim  to  control.  One  of  the  major  barriers  to  consensus  on  these  two  factors  was  an 
observed  variation  by  region  and  differences  in  COIN  force  and  government  activities  between 
rural  areas  and  major  population  centers.  These  disputes  highlight  not  only  the  real  differences 
between  COIN  efforts  in  population  centers  versus  rural  areas  in  2013  Afghanistan  but  also 
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the  broader  historical  fact  that  the  detailed  application  and  progress  of  various  COIN  efforts 
can  vary  considerably  across  the  different  regions  of  a  country  facing  an  insurgency. 

Notes  on  Factors  Present  or  Absent  in  2013  but  Tenuous  in  the  Future 

As  noted  earlier,  the  discussion  during  the  expert  elicitation  exercise  often  highlighted  factors 
that  might  become  precarious  in  the  future  as  international  forces  and  support  draw  down, 
regardless  of  whether  one  or  more  panelists  asserted  these  factors  as  favorably  present  or  absent 
in  Afghanistan  in  early  2013.  The  following  factors  received  at  least  some  mention  of  future 
concerns  about  their  sustainability  in  the  face  of  ISAF  retrograde: 

•  Themes  and  messages  coordinated  for  all  involved  government  agencies  (consensus 
absent). 

•  Insurgents’  ability  to  replenish  resources  significantly  diminished  (consensus  absent). 

•  COIN  force  efforts  resulting  in  increased  costs  for  insurgent  processes  (consensus  pres¬ 
ent). 

•  COIN  forces  effectively  disrupt  insurgent  recruiting  (consensus  absent). 

•  Intelligence  adequate  to  support  kill/capture  or  engagements  with  insurgents  on  COIN 
force’s  terms  (no  consensus). 

•  COIN  force  of  sufficient  strength  to  force  insurgents  to  fight  as  guerrillas  (consensus  pres¬ 
ent). 

•  Unity  of  effort/ unity  of  command  maintained  (consensus  absent). 

•  Short-term  investments,  improvements  in  infrastructure/development,  or  property  reform 
in  area  of  conflict  controlled  or  claimed  by  COIN  force  (consensus  present). 

•  COIN  force  establishes  and  then  expands  secure  areas  (no  consensus). 

•  Government/COIN  reconstruction/development  efforts  sought/achieved  improvements 
substantially  above  historical  baseline  (no  consensus). 

•  COIN  force  provides  or  ensures  provision  of  basic  services  in  areas  it  controls  or  claims 
to  control  (no  consensus). 

•  Insurgent  force  individually  superior  to  COIN  force  by  being  either  more  professional  or 
better  motivated  (no  consensus). 

This  finding  is  concerning  because  the  current  scorecard  balance  of  eight  good  factors,  six 
bad  factors,  and  an  overall  score  of  +2,  with  attendant  bands  of  uncertainty,  indicates  a  need 
for  further  improvement.  These  expressed  concerns  cast  significant  doubt  on  prospects  for 
further  improvement  once  ISAF  withdraws  and,  in  fact,  imply  a  significant  threat  of  sliding 
backward  toward  even  less  optimistic  scores. 

Afghanistan  in  Comparison  to  Specific  Historical  Cases 

The  analysis  thus  far  has  compared  Afghanistan  in  early  2013  with  patterns  of  data  observed  in 
the  59  historical  cases  or  with  those  cases  in  the  aggregate.  Is  Afghanistan  significantly  similar 
to  any  specific  historical  case?  The  short  answer:  not  really. 

The  case  of  Afghanistan  in  early  2013  is  not  particularly  similar  in  its  specific  details 
to  any  of  the  59  historical  cases.  In  terms  of  its  good-and-bad-factor  summary  (see  Table  5), 
Afghanistan  (8  -  6  =  2)  is  not  especially  similar  to  any  case.  Two  cases  have  total  scores  of  +2, 
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Iraqi  Kurdistan  (4-2  =  2)  and  Western  Sahara  (4-2  =  2).  While  four  cases  have  a  total  of 
eight  good  factors  present,  they  have  fewer  bad  factors.  All  of  the  cases  with  six  bad  factors 
have  three  of  fewer  good  factors  and  negative  overall  scores,  and  they  were  COIN  losses.  The 
single  case  with  the  greatest  degree  of  similarity  at  the  scorecard  level  is  Guatemala  (8  -  4  =  4), 
though  it  is  not  at  all  similar  in  the  specific  pattern  of  detailed  scorecard  factors  present  or 
absent  or  in  its  case  narrative. 

Considering  all  phases  of  all  cases,12  and  examining  the  extent  to  which  the  34  consensus 
factors  and  subfactors  for  Afghanistan’s  scorecard  match  up,  is  a  better  way  to  assess  the  simi¬ 
larity  between  the  historical  cases  and  the  current  Afghanistan  case.13  Following  this  proce¬ 
dure  reveals  a  range  of  similarity.  The  least  similar  phase  of  all  of  the  historical  cases  matched 
on  11  of  34  scored  factors  (far  below  the  17  one  would  expect  by  random  chance  on  binary 
variables),  and  the  closest  match  was  26  of  34  points  in  common.  Afghanistan  in  early  2013 
was  not  particularly  like  any  of  the  phases  of  previous  Afghanistan  cases  by  this  metric.  (Phases 
of  previous  insurgencies  in  Afghanistan  scored  17,  20,  21,  21,  21,  22,  23,  23,  and  23.)  This  is, 
perhaps,  good  news,  as  both  previous  insurgencies  in  Afghanistan  were  won  by  the  insurgents, 
so  one  would  hope  that  this  case  differs! 

Even  considering  cases  with  a  greater  degree  of  similarity  (phases  of  cases  matching  on  as 
many  as  26  of  the  34  scored  factors)  in  specific  scorecard  scores  does  not  provide  a  compari¬ 
son  that  is  more  than  passingly  similar  in  its  narrative.  Thus,  we  make  no  specific  comparison 
between  Afghanistan  and  any  individual  case,  but  we  do  compare  Afghanistan  on  what  all 
59  core  historical  cases  shared,  the  scorecard. 

2013  Results  Compared  with  2011  Results 

As  noted,  this  report  repeats  an  earlier  effort  to  complete  the  COIN  scorecard  for  Afghani¬ 
stan,  findings  from  which  were  published  in  Counterinsurgency  Scorecard:  Afghanistan  in  Early 
2011  Relative  to  the  Insurgencies  of  the  Past  30  Years  and  based  on  research  reported  in  Victory 
Has  a  Thousand  Fathers:  Sources  of  Success  in  Counterinsurgency. 1 4  The  results  of  this  exercise 
are  different  from  those  of  the  original,  both  because  two  years  have  passed  and  conditions 
have  changed  in  Afghanistan  and  because  the  scorecard  itself  has  been  updated  to  account  for 
a  larger  number  of  cases  and  longer  period  of  history.  While  the  two  sets  of  scorecard  results 
cannot  be  directly  compared,  they  can  be  compared  obliquely.  After  all,  both  versions  of  the 
scorecard  perfectly  discriminate  the  sets  of  historical  cases  on  which  they  were  based  into  wins 
and  losses,  and  both  have  an  observed  “empirical  gap”  separating  the  highest-scoring  win  from 
the  lowest-scoring  loss — a  gap  of  uncertainty  between  the  outcomes  actually  observed. 

In  the  2011  Afghanistan  scoring,  the  scorecard  result  was  found  to  be  in  that  gap  between 
historical  wins  and  losses.  In  the  2013  scoring  reported  here,  the  result  has  crossed  that  gap 
and  now,  at  +2,  is  among  the  historical  COIN  winners  (though  equal  to  the  lowest-scoring 
winners  and  with  a  band  of  uncertainty  due  to  nonconsensus  half-point  scores,  from  -2  to  +6, 


12  In  Paths  to  Victory ,  each  case  was  divided  into  between  two  and  five  phases,  for  a  total  of  205  phases.  In  the  context  of 
this  study,  data  for  each  case  refer  to  the  decisive  (and  usually  terminal)  phase. 

13  Although  the  panel  was  asked  to  score  42  individual  factors  as  present  or  absent,  consensus  was  reached  for  only  34.  The 
eight  factors  on  which  there  was  not  consensus  cannot  be  assessed  as  like  or  unlike  the  scores  for  historical  cases. 

^  Paul  2011;  Paul,  Clarke,  and  Grill,  2010. 
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covering  both  the  lowest-scoring  historical  wins  and  the  highest-scoring  historical  losses).  This 
is  consonant  with  the  general  feedback  from  expert  panelists  in  the  2013  exercise:  Participants 
conveyed  that  several  important  areas  had  seen  improvement  since  the  2010-2011  time  frame. 

The  2011  report  highlighted  three  critical  areas  for  improvement:  (1)  the  competency, 
legitimacy,  and  popular  support  of  the  Afghan  government;  (2)  security;  and  (3)  disrupting 
tangible  support  to  the  insurgents.  The  current  report  highlights  only  two  critical  areas  for 
improvement,  one  of  which  matches  the  previous  study:  (1)  disrupting  tangible  support  to  the 
insurgents,  and  (2)  demonstration  of  commitment  and  motivation  on  the  part  of  the  Afghan 
government  and  security  forces.  Compared  with  the  2011  report,  this  report  is  much  more 
cognizant  of  the  impending  departure  of  the  majority  of  coalition  forces  and  the  correspond¬ 
ing  challenges  to  prospects  for  maintaining  the  good  practices  that  were  in  place  at  the  time 
of  this  study. 

Scorecard  and  Duration  of  Conflict 

This  comparison — this  transition  from  within  the  uncertainty  gap  to  among  the  historical 
winners  between  2011  and  2013 — becomes  very  interesting  in  light  of  other  findings  con¬ 
cerning  the  scorecard  from  Paths  to  Victory.  When  we  briefed  results  from  the  original  Victory 
Has  a  Thousand  Fathers  scorecard,  we  would  often  be  asked  something  like,  “How  long  does 
the  COIN  force  need  to  keep  a  strong  scorecard  balance  before  it  wins?”  We  were  unable  to 
answer  this  question  with  data  from  Victory  Has  a  Thousand  Fathers,  but  we  could  do  so  with 
the  new  data  collected  for  Paths  to  Victory.  The  answer  is,  on  average  (median),  5.75  years.  That 
is,  in  the  28  insurgencies  won  by  governments  since  WWII,  all  had  scorecard  balances  of  +2  or 
greater  by  the  end  of  the  conflict,  and,  on  average,  the  conflict  ended  5.75  years  after  achiev¬ 
ing  a  scorecard  balance  of  +2  or  greater.  There  was  considerable  variation  in  this  average,  with 
some  cases  having  a  positive  scorecard  balance  for  only  a  short  time  before  a  decisive  outcome 
and  others  having  a  positive  scorecard  balance  for  the  entirety  of  a  conflict  running  12  years  or 
more.  But  the  average  was  5.75  years. 

This  has  important  implications  for  Afghanistan.  If  we  impute  the  2011  results  as  falling 
short  of  the  needed  threshold  and  consider  the  2013  results  as  finally  being  a  minimally  suf¬ 
ficient  scorecard  balance,  then  history  suggests  that  a  positive  scorecard  balance  would  need  to 
be  sustained  for  several  more  years  (5.75  on  average,  but  perhaps  much  longer)  before  a  conclu¬ 
sion  favoring  the  government.  While  this  does  suggest  that  COIN  efforts  in  Afghanistan  are 
on  the  right  path,  the  concerns  raised  about  the  ability  of  the  Afghan  government  and  security 
forces  to  sustain  some  of  the  good  practices  after  the  departure  of  the  majority  of  international 
forces  call  into  question  whether  the  Afghans  can  maintain  a  positive  scorecard  balance  by 
themselves  and  ultimately  prevail  over  the  insurgency. 


Conclusions  and  Recommendations 

Comparing  Afghanistan  in  early  2013  with  Afghanistan  in  early  2011  and  with  the  historical 
insurgency  cases  since  WWII  offers  some  support  for  optimism,  but  it  also  raises  some  sig¬ 
nificant  concerns.  When  using  a  scorecard  of  15  good  factors  and  11  bad  factors  based  on  the 
historical  record,  Afghanistan’s  current  balance  of  +2  places  it  among  the  historical  winners. 
Further,  Afghanistan’s  current  score  of  eight  of  15  positive  factors  is  strong  relative  to  other 
historical  winners. 
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However,  on  the  concerning  side,  the  band  of  uncertainty  around  the  total  score  includes 
both  historical  wins  and  historical  losses,  and  the  current  score  of  six  bad  factors  is  larger  than 
the  number  of  negative  factors  possessed  by  any  of  the  winners.  Furthermore,  although  we 
can  currently  count  eight  good  factors  as  present,  two  factors  that  were  present  in  all  historical 
wins  are  missing:  the  disruption  of  insurgents’  tangible  support  and  a  clear  demonstration  of 
commitment  and  motivation  on  the  part  of  the  host-nation  government  and  indigenous  secu¬ 
rity  forces.  Two  final  concerns  raised  by  the  experts  who  participated  in  the  scoring  were  that 
the  Afghans  may  not  be  able  to  maintain  several  of  the  good  practices  by  themselves  once  the 
coalition  of  international  supporters  withdraws  the  preponderance  of  its  forces  and  that  sev¬ 
eral  good  COIN  practices  appear  to  be  present  in  certain  (densely  populated  or  uncontested) 
regions  of  the  country  and  absent  in  others. 

Thus,  this  report  highlights  two  critical  areas  for  improvement:  (1)  the  disruption  of  tan¬ 
gible  support  to  the  insurgents  and  (2)  a  demonstration  (and  improvement)  of  commitment 
and  motivation  on  the  part  of  the  Afghan  government  and  Afghan  security  forces. 


APPENDIX 


Details  of  the  Expert  Elicitation 


The  Delphi  Method 

The  Delphi  method  was  developed  at  RAND  in  the  1960s.  While  the  technique  has  been 
refined  over  the  years,1  the  fundamental  premise  remains  the  same.  Experts  individually  make 
assessments  or  provide  input  and  then  offer  written  justification  for  those  assessments.  These 
experts  are  then  given  the  opportunity  to  privately  review  the  justifications  offered  by  other 
participants  and  revise  their  assessments  based  on  lines  of  reasoning  that  they  had  failed  to 
include  in  their  own  initial  calculations.  The  result  is  a  consensual  set  of  expert  assessments 
based  on  more  information  than  any  single  expert  initially  considered.  Because  participants 
work  in  private  and  remain  anonymous  to  each  other,  final  evaluations  are  reached  without  any 
of  the  psychological  pitfalls  of  committee  work,  such  as  “specious  persuasion,  the  unwilling¬ 
ness  to  abandon  publicly  expressed  opinions,  and  the  bandwagon  effect  of  majority  opinion.”2 

A  simplified  example  of  a  Delphi  exercise  demonstrates  this  logic.  Imagine  that  a  Delphi 
exercise  is  convened  to  assure  victory  in  a  carnival  game:  The  investigators  wish  to  know  how 
many  peanuts  are  contained  in  a  large  glass  pig.  A  panel  of  experts  is  assembled,  including 
(among  others)  a  physicist,  a  mathematician,  a  material  scientist,  a  statistician,  and  an  expert 
in  the  history  of  mountebanks.  Each  performs  his  or  her  calculations  and  generates  an  estimate 
of  the  peanut  content  of  the  pig.  Then,  each  is  asked  to  justify  his  or  her  response,  explaining 
the  calculations  involved.  One  participant  might  begin  with  the  formula  for  the  volume  of  an 
ellipsoid  and  then  assume  a  volume  for  peanuts  and  proceed.  Another  might  begin  with  the 
volume  of  an  ellipsoid  and  then  add  a  clever  correction  factor  for  the  additional  volume  repre¬ 
sented  by  the  pig’s  feet  and  head.  Yet  another  might  simply  use  the  volume  of  a  sphere  but  add 
an  innovative  adjustment  for  the  stochastic  nature  of  the  space  between  peanuts  as  they  do  or 
do  not  nest  well  with  each  other.  The  expert  on  mountebanks  may  not  able  to  articulate  his  or 
her  volume  calculation  well  at  all  but  may  make  two  critical  observations  about  the  kinds  of 
tricks  that  carnival  operators  are  likely  to  pull  to  make  such  estimation  difficult — say,  incon¬ 
sistent  thickness  of  the  glass  of  the  pig  or  the  use  of  peanuts  of  different  sizes.  As  the  experts 
review  the  justifications  and  calculations  made  by  the  others,  they  may  recognize  factors  that 
they  failed  to  include  in  their  own  calculations  or  come  to  understand  that  they  have  over-  or 
underestimated  some  critical  quantity.  The  revised  estimates  are  likely  to  be  based  on  more 


1  See,  for  example,  Carolyn  Wong,  How  Will  the  e-Explosion  Affect  How  We  Do  Research l  Phase  I:  The  E-DEL+I  Proof-of- 
Concept  Exercise,  Santa  Monica,  Calif.:  RAND  Corporation,  DB-399-RC,  2003. 

2  Bernice  B.  Brown,  Delphi  Process:  A  Methodology  Used  for  the  Elicitation  of  Opinions  of  Experts,  Santa  Monica,  Calif.: 
RAND  Corporation,  P-3923,  1968,  p.  2. 
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complex  calculations,  be  better  calculations,  and  be  closer  to  each  other  than  were  the  initial 
individual  expert  estimates. 


The  RAND  Afghanistan  Delphi  Exercise 

The  RAND  Afghanistan  Delphi  exercise  was  an  iterative  Delphi  exercise  based  on  the  classic 
model.  It  was  completed  via  iterative  email  exchange  between  April  16  and  May  20,  2013.  This 
section  details  the  process  used. 

By  definition,  an  expert  elicitation  is  only  as  good  as  the  experts  elicited.  An  initial  list  of 
candidate  participants  was  generated  in  consultation  with  senior  RAND  managers  and  based 
on  participants  in  the  previous  exercise,  documented  in  Counterinsurgency  Scorecard:  Afghani¬ 
stan  in  Early  2011  Relative  to  the  Insurgencies  of  the  Past  30  Years  (OP-337-OSD).3  This  list  was 
expanded  in  consultation  with  the  project  sponsor  and  based  on  interactions  in  early  briefings 
of  draft  results  from  Paths  to  Victory :  Lessons  from  Modern  Insurgencies  (RR-291/1-OSD).4  An 
initial  list  of  26  candidates  emerged  from  this  process.  Of  this  group,  21  initially  agreed  to  par¬ 
ticipate.  Of  those  21,  20  completed  the  first  round  of  the  exercise,  with  one  declining  to  partici¬ 
pate  when  the  details  of  the  exercise  became  clearer.  Of  the  20  initial  participants,  19  completed 
the  entire  exercise,  with  one  further  withdrawal  due  to  time  constraints.  Participants  included 
RAND  staff  with  expertise  on  Afghanistan  or  who  had  deployed  to  Afghanistan;  serving  field- 
grade  U.S.  military  officers  with  multiple  (and  recent)  deployments  to  Afghanistan;  military 
veterans  who  were  current  on  COIN  research;  defense  civilian  representatives  from  the  Office 
of  the  Secretary  of  Defense,  the  Joint  Staff,  and  U.S.  Central  Command;  government  civilians 
with  experience  in  Afghanistan;  staff  from  the  U.S.  Agency  for  International  Development; 
university  faculty  members;  journalists;  and  experts  from  other  prominent  think  tanks. 

A  summary  set  of  instructions  defined  each  factor  more  extensively  and  gave  general 
guidance  regarding  the  exercise.  This  guidance  included  the  following  text: 

Please  evaluate  each  factor  as  present  (1)  or  absent  (0)  in  contemporary  Afghanistan.  All 
questions  pertain  to  the  area  of  conflict  unless  otherwise  specified.  If  rhe  answer  is  “well, 
it  depends  on  where  specifically  you  consider — that  factor  is  present  in  part  of  the  country 
but  absent  in  another  part,”  please  make  “ worst-case ”  assessments.  That  is,  if  something  is 
going  well  in  RC  [Regional  Command]  South  but  poorly  in  RC  East,  make  your  scoring 
based  on  RC  East. 

A  caveat:  While  we  want  “worst-case”  assessments,  we  do  not  want  assessments  driven  by 
isolated  events.  If  a  positive  factor  is  present  over  the  vast  majority  of  the  area  of  conflict, 
score  it  as  present  even  if  there  are  one  or  two  isolated  incidents  where  the  factor  was  not 
present. 

The  iterative  Delphi  exercise  included  four  scoring  rounds  with  two  phases  in  each  round. 
Because  it  was  clear  that  positions  had  become  entrenched,  and  out  of  respect  for  participants’ 
time,  the  second  (discussion-only)  phase  of  the  third  round  was  canceled  by  the  moderator, 
progressing  the  exercise  directly  to  the  final  phase.  This  resulted  in  a  total  of  six  phases.  In  the 


3  Paul,  2011. 

4  Paul,  Clarke,  Grill,  and  Dunigan,  2013. 
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first  phase  of  each  round,  participants  provided  scores  for  each  factor,  indicating  whether  they 
believed  it  was  present  (“1”)  or  absent  (“0”)  in  Afghanistan.  In  the  second  phase  of  each  round, 
participants  were  shown  their  scores  relative  to  the  mean  scores  of  all  participants.  In  all  phases 
save  the  very  first  and  the  very  last,  participants  were  asked  to  justify  their  minority  positions 
(factors  on  which  they  deviated  from  the  group  mean  by  0.4  or  more)  and  contribute  to  the 
ongoing  discussion  about  the  presence  or  absence  of  the  factors. 

In  each  phase  save  the  first  (nothing  to  discuss  yet)  and  the  last  (discussion  concluded), 
participants  were  asked  to  contribute  to  the  discussion.  In  a  traditional  Delphi  exercise,  scor¬ 
ers  are  asked  to  justify  all  their  ratings  or  calculations  in  the  first  round.  However,  because 
this  exercise  included  42  individual  factors  and  because  all  participants  volunteered  their  time, 
participants  were  asked  to  provide  justification  only  for  minority  positions,  lest  a  great  quan¬ 
tity  of  volunteered  time  be  consumed  generating  justifications  for  positions  about  which  the 
entire  panel  was  in  complete  agreement.  In  the  second  phase  of  each  round,  participants  whose 
score  on  a  factor  differed  from  the  group  mean  by  0.4  or  more  were  informed  that  theirs  was 
a  minority  position  and  asked  to  justify  it.  In  this  way,  the  discussion  remained  focused  on 
factors  that  were  actually  contentious,  rather  than  being  diluted  with  justifications  of  factors 
about  which  there  was  already  significant  concordance.  Scores  that  became  minority  positions 
in  subsequent  rounds  (due  to  either  changed  scores  or  movement  of  the  mean)  were  flagged  as 
newly  minority  positions,  indicating  that  a  new  justification  was  required  from  the  participant. 

After  responding  to  requests  for  justification  of  their  minority  positions,  participants  were 
asked  to  weigh  in  on  any  of  the  ongoing  discussion  of  any  of  the  factors.  Space  was  made  avail¬ 
able  for  written  rebuttals,  counterarguments,  endorsements,  and  so  on,  aimed  at  initial  minor¬ 
ity  defenses  or  at  discussion  ensuing  from  them.  No  limit  was  placed  on  the  volume  or  charac¬ 
ter  of  the  discussion,  though  participants  were  encouraged  to  be  concise.  Instructions  invited 
participants  to  refer  to  studies,  data  sets,  personal  experiences,  or  other  evidence  that  they  felt 
supported  their  positions  or  otherwise  accounted  for  their  reasoning  or  logic.  The  discussion 
was  considerable,  spanning  more  than  60  single-spaced  pages  in  aggregate. 


Raw  Delphi  Scores 

Table  A.l  presents  the  average  score  for  each  factor  across  the  19  participants  who  ultimately 
completed  the  exercise.  Because  the  scoring  was  binary  (0  or  1),  the  raw  average  can  be  accu¬ 
rately  interpreted  as  the  proportion  of  participants  who  indicated  that  a  factor  was  present  in 
their  final  scoring.  For  example,  the  first  factor  is  “COIN  force  and  government  actions  con¬ 
sistent  with  messages  (delivering  on  promises).”  The  raw  average  is  0.16,  which  indicates  that 
16  percent  of  participants  indicated  that  the  factor  was  present  (scored  it  as  “1”)  in  their  final 
scoring. 

For  reference,  Table  A.l  also  presents  the  consensus  results  (in  the  “Rounded  Result” 
column)  used  in  the  main  body  of  this  report.  Recall  that  70-percent  agreement  was  the 
threshold  specified  for  consensus.  This  means  that  raw  average  scores  of  0.7  or  higher  were 
considered  to  indicate  consensus  on  presence  and  were  rounded  to  “1”  accordingly.  Inversely, 
scores  of  0.3  or  lower  were  rounded  to  consensus  absence,  or  “0.”  Scores  between  0.3  and  0.7 
were  left  as  contentious  scores  and  rounded  to  0.5,  indicating  a  lack  of  agreement  and  neither 
presence  nor  absence. 
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Table  A.1 

Raw  Average  Scores  from  RAND  Afghanistan  Delphi  Exercise  and  Rounded  Consensus  Results 

Raw  Rounded 

Factor  Average  Result 

COIN  force  and  government  actions  consistent  with  messages  (delivering  on  promises)  0.16  0 

Forces  of  order  maintains  credibility  with  populations  in  the  area  of  conflict  (includes  0.16  0 

expectation  management) 

Messages/themes  coherent  with  overall  COIN  approach  0.58  0.5 

Forces  of  order  avoids  creating  unattainable  expectations  0.21  0 

Themes  and  messages  coordinated  for  all  involved  government  agencies  0.26  0 

Flow  of  cross-border  insurgent  support  across  significantly  decreased,  remains  0  0 

dramatically  reduced,  or  largely  absent 

Important  external  support  to  insurgents  significantly  reduced  0  0 

Important  internal  support  to  insurgents  significantly  reduced  0.21  0 

Insurgents'  ability  to  replenish  resources  significantly  diminished!  0.11  0 

Insurgents  unable  to  maintain  or  grow  force  size  0.26  0 

COIN  force  efforts  resulting  in  increased  costs  for  insurgent  processes  0.95  1 

COIN  forces  effectively  disrupt  insurgent  recruiting  0.16  0 

COIN  forces  effectively  disrupt  insurgent  materiel  acquisition  0.21  0 

COIN  forces  effectively  disrupt  insurgent  intelligence  0.11  0 

COIN  forces  effectively  disrupt  insurgent  financing  0.16  0 

Government  leaders  selected  in  a  manner  considered  just  and  fair  by  majority  of  0.11  0 

population  in  the  area  of  conflict 

Majority  of  citizens  in  the  area  of  conflict  view  government  as  legitimate  0.11  0 

Government  corruption  reduced/good  governance  increased  since  onset  of  conflict  0  0 

Intelligence  adequate  to  support  kill/capture  or  engagements  on  COIN  force's  terms  0.63  0.5 

Intelligence  adequate  to  allow  COIN  force  to  disrupt  insurgent  processes  or  operations  0.89  1 

COIN  force  of  sufficient  strength  to  force  insurgents  to  fight  as  guerrillas  0.89  1 

Unity  of  effort/unity  of  command  maintained  0.26  0 

COIN  force  avoids  excessive  collateral  damage,  disproportionate  use  of  force,  or  other  0.74  1 

illegitimate  applications  of  force 

COIN  force  seeks  to  engage  and  establish  positive  relations  with  population  in  area  of  0.89  1 

conflict 

Short-term  investments,  improvements  in  infrastructure/development,  or  property  0.95  1 

reform  in  area  of  conflict  controlled  or  claimed  by  COIN  force 

Majority  of  population  in  area  of  conflict  supports/favors  COIN  forces  0.89  1 

COIN  force  establishes  and  then  expands  secure  areas  0.37  0.5 

Government/COIN  reconstruction/development  sought/achieved  improvements  0.63  0.5 

substantially  above  historical  baseline 

COIN  force  provides  or  ensures  provision  of  basic  services  in  areas  it  controls  or  claims  0.37  0.5 

to  control 

Perception  of  security  created  or  maintained  among  population  in  areas  COIN  force  0.53  0.5 

claims  to  control 

COIN  force  employs  escalating  repression  0  0 

COIN  force  employs  collective  punishment  0  0 

Government  involves  corrupt  and  arbitrary  personalistic  rule  1  1 
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Table  A.1 — Continued 

Raw  Rounded 

Factor  Average  Result 

Country  elites  have  perverse  incentives  to  continue  conflict  0.89  1 

External  professional  military  forces  are  engaged  in  fighting  on  behalf  of  the  0.11  0 

insurgents 

Host  nation  is  economically  dependent  on  external  supporters  1  1 

Fighting  is  primarily  initiated  by  the  insurgents  0.79  1 

COIN  force  fails  to  adapt  to  changes  in  adversary  strategy,  operations,  or  tactics  0.16  0 

COIN  force  engages  in  more  coercion/intimidation  than  insurgents  0  0 

Insurgent  force  individually  superior  to  COIN  force  by  being  either  more  professional  or  0.32  0.5 

better  motivated 

Forces  of  order  or  allies  rely  on  looting  for  sustainment  0.32  0.5 

COIN  force  and  government  have  different  goals/levels  of  commitment  1  1 
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The  RAND  report  Paths  to  Victory:  Lessons  from  Modern  Insurgencies  added  41  new  cases  to  a  previously  studied  set 
of  30  insurgencies,  examining  the  71  insurgencies  begun  and  completed  worldwide  between  World  War  II  and  2008 
to  analyze  correlates  of  success  in  counterinsurgency  (COIN).  A  key  finding  of  this  research  was  that  a  case's  score 
on  a  scorecard  of  15  equally  weighted  good  and  11  equally  weighted  bad  COIN  factors  and  practices  perfectly 
discriminated  the  outcomes  of  the  cases  analyzed.  That  is,  the  balance  of  good  and  bad  factors  and  practices 
correlated  with  either  a  COIN  win  (insurgency  loss)  or  a  COIN  loss  (insurgency  win)  in  the  overall  case.  Using  the 
scorecard  approach  as  its  foundation,  a  RAND  study  sought  to  apply  the  findings  to  the  case  of  Afghanistan  in  early 
2013.  The  effort  involved  an  expert  elicitation,  or  Delphi  exercise,  in  which  experts  were  asked  to  make  "worst-case" 
assessments  of  the  factors  to  complete  the  scorecard  for  ongoing  operations  in  Afghanistan.  The  consensus  results 
revealed  that  early  2013  Afghanistan  ranks  among  the  historical  COIN  winners,  but  its  score  is  equal  to  those  of  the 
lowest-scoring  historical  wins.  This  tenuous  position  points  to  several  areas  in  need  of  improvement,  but  particularly 
the  need  to  disrupt  the  flow  of  insurgent  support  and  the  need  for  the  Afghan  government  and  Afghan  security 
forces  to  better  demonstrate  their  commitment  and  motivation.  Afghanistan  in  early  2011  scored  in  the  middle  of 
the  historical  record  in  terms  of  COIN  wins  and  losses,  suggesting  an  overall  improvement  in  COIN  progress  in  that 
conflict  by  early  2013.  However,  conditions  may  change  as  coalition  forces  prepare  to  hand  over  responsibility  for 
the  country's  security  to  the  Afghan  government  and  Afghan  security  forces  in  2014. 
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